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CHAPTER  6 

LARGE  SCALE  PACKET  SWITCHED  NETWORK  DESIGN  TRADEOFFS 
6. i(  INTRODUCTION 

f 

Previous  NAC  data  network  studies^  ( [NAC , 1975],  [NAC,  1976]) 
have  revealed  the  fundamental  properties  and  relative  costs  of  a 
variety  of  network  architecture  alternatives.  The  cost  comparisons 
were  made  with  a constant  backbone  subnet  average  end-to-end 
delay  - a crucial  performance  parameter.  Complex  line  cost 
structures  were  created  to  accurately  model  existing  and  projected 
tariffs . 

This  study  focuses  on  the  packet-switched  network  approach 
with  all  users  fully  integrated  in  an  overall  facility  sharing 
plan.  Simplifying  assumptions  are  made  throughout  so  that  funda- 
mental design  decision  tradeoffs  can  be  revealed  and  so  that  the 
sensitivities  to  cost  trends  brought  about  by  new  technologies 
can  be  assessed . ipln  particular,  the  effects  of  the  following  anti- 
cipated developments  are  exposed: 

• Use  of  satellite  transmission. 

• Different  switch  capabilities. 


Altered  ratio  of  processor  cost  to  transmission 
line  cost. 


This  study  also  extends  the  results  of  previous  work  by 
dealing  with  much  higher  traffic  throughput  volumes  - 1.8  MBS  to 
180  MBS  -and  by  using  a more  accurate  data  base  of  anticipated 
military  data  message  traffic  than  was  previously  available. 
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6.2  THE  NETWORK  MODEL 


The  network  model  assumes  that  users  in  the  same  physical 
location  merge  their  traffic  with  modem  sharing  units  (MSU’s), 
terminal  control  units  (TCU's),  multiplexers,  concentrators,  or 
some  combination  of  these  devices  so  that  the  entire  data  stream 
in  and  out  of  a location  appears  as  a single  stream  to  the  network. 
Thus  computer  and  terminal  traffic,  and  traffic  belonging  to  dif- 
ferent systems  are  summed  indifferently  for  each  site  to  create  a 
single  lumped  external  data  I/O  requirement.  In  practice,  any 
site  where  such  a merge  would  be  impractical  or  inadvisable  could 
opt  to  have  separate  multiple  access  facilities  and  incur  the 
additional  cost  with  minimal  consequence  to  other  network  users. 
Thus  for  the  model  used  in  this  study,  all  configuration  considera- 
tions within  a facility  are  ignored.  The  data  base  contained 
instances  of  facilities  that  were  separately  identified  but  had 
geographic  coordinates  indicating  colocation.  These  facilities 
were  treated  as  separate  data  streams,  but  in  cases  where  tele- 
communication connections  were  required  between  them,  no  cost  was 
assigned. 

The  network  architecture  adopted  for  this  study  of  packet 
switching  network  costs  is  a three-tiered  hierarchical  system. 

As  already  mentioned,  at  the  hierarchy  level  closest  to  the  user, 
an  optimal  local  site  configuration  is  assumed  so  that  the  data 
traffic  into  and  out  of  the  site  appears  to  the  network  as  a 
single  stream.  At  the  hierarchy  level  furthest  from  the  user,  a 
backbone  network  provides  high  interconnection,  good  utilization 
of  generally  high  bandwidth  links,  alternate  paths  for  throughput 
improvement  (through  routing  optimization)  and  reliability.  To 
insure  adequate  backbone  reliability  at  least  two  node  disjoint 
paths  are  provided  between  all  points  in  the  backbone  network, 
except  for  some  designs  which  used  satellite  links. 
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An  intermediate  hierarchy  level,  called  concentration,  is 
introduced  between  the  local  sites  and  the  backbone  network. 

These  intermediate  devices  are  generically  called  concentrators, 
but  in  practice  may  be  concentrators,  multiplexers,  or  statistical 
multiplexers  according  to  cost/performance  requirements.  Their 
function  is  to  collect  the  traffic  from  a regional  group  of  sites 
and  lump  it  into  a single  channel,  thus  reducing  the  costs  for 
these  sites  to  access  the  backbone  subnet. 

The  concentrators  were  assumed  to  be  limited  to  a throughput 
of  230  KBS  - so  they  function  only  to  merge  the  traffic  streams 
of  relatively  small  volume  sites.  Consequently  their  use  diminishes 
as  traffic  volumes  are  scaled  up.  They  serve  to  reduce  the 
influence  of  the  smaller  volume  sites  on  the  overall  network 


strategy.  Higher  throughput  concentration  points  are  not  viable 
because  of  reliability  considerations  and  also  because  the  addi- 
tional cost  of  installing  a full  backbone  switch  is  small. 

Sites  are  connected  directly  to  a backbone  switch  or  in  some 
cases  go  through  a concentrator  to  the  backbone  switch  - both 
star  or  centralized  configurations.  Multipointing  was  not  permitted 
since  this  would  presume  a collection  of  sites  willing  to  operate 
at  a common  speed  and  discipline  and  willing  to  surrender  their 
independent  control  of  the  local  access  portion  of  their  end-to- 
end  connections.  Multipointing  would  also  give  diminished  and 
unequal  reliability  to  the  served  sites. 

Since  we  are  dealing  with  high  traffic  levels,  the  throughput 
requirements  in  the  backbone  network  lead  to  high  capacity  lines  - 
and  consequently  low  transmission  delays  - in  the  backbone  portion 
of  an  end-to-end  connection.  For  the  traffic  levels  considered 


here, -these  backbone  delays  are  generally  far  below  any  interactive 

> 

« or  other  real-time  data  requirements.  The  delays  in  the  local 

access  portions  may  be  more  substantial,  particularly  for  low 
speed  access  lines. 
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requirements  for  the  users  at  each  site,  by  using  the  star  configura- 
tion, no  assumptions  had  to  be  made.  Each  site  directly  connected 
to  a backbone  switch  can,  in  practice,  upgrade  its  access  line 
without  impacting  any  other  feature  of  the  design.  Sites  that 
are  connected  to  a concentrator,  on  the  other  hand,  share  the  use 
of  the  concentrator-to-backbone  connection.  However  the  delay 
encountered  over  this  second  leg  of  the  local  access  is  usually  a 
small  fraction  of  the  total  local  access  delay;  so  here  again, 
the  individual  user  is  in  substantial  control  of  his  delay  perfor- 
mance . 

By  the  same  logic,  users  requiring  isolation  from  the  common 
data  stream,  for  any  reason,  can  build  separate  access  facilities 
and  incur  the  extra  cost  without  influencing  the  remainder  of  the 
design  (except  port  facilities) . If  this  practice  became  widespread, 
however,  it  would  influence  the  optimal  number  and  location  of  back- 
bone and/or  concentration  facilities  and  cause  a general  increase  in 
network  costs  - a result  of  trying  to  reduce  the  cost  burden  of  the 
special  access  requirements.  This  accommodation  cost  would  be  an 
implicit  part  of  the  global  network  design  cost  and  would  be 
difficult  to  apportion  to  individual  special  access  users. 

Sites  accessing  the  network  through  a concentrator  do  suffer 
some  diminished  reliability  compared  to  users  with  direct  backbone 
access.  Even  if  relative  equipment  reliability  is  not  an  issue, 
the  backbone  nodes  have  guaranteed  greater  connectivity  than  is 
provided  by  a single  concentrator-to-backbone  link.  To  compensate 
for  this  we  calculated  the  additional  cost  of  dual  homing  the 
concentrators  - although  no  attempt  was  made  to  reoptimize  the 
network  on  the  basis  of  dual  homing  considerations.  Thus  the 
dual  homing  cost  is  somewhat  of  an  overestimate  of  the  actual 
added  cost.  Dual  homing  provides  a dedicated  backup  connection 
to  an  alternate  backbone  switch,  but  this  backup  is  presumed  to 
be  idle  (except  for  periodic  testing,  etc.)  during  normal  operation. 
In  fact,  it  is  the  inability  of  the  concentrator  to  route  simulta- 
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neously  over  alternate  paths  that  fundamentally  distinguishes  it 
from  a backbone  switch. 

All  user  sites  are  considered  to  be  potential  concentrator  and 
backbone  switch  locations.  In  practice  such  sites  may  be  restrict- 
ed, for  example,  to  specially  secure  facilities  or  where  maintenance 
personnel  can  be  conveniently  colocated.  We  did  not  allow  new  nodes 
to  be  introduced  because  of  their  advantage  as  a concentrator  or 
switch  site.  Given  the  geographical  dispersion  of  the  existing 
sites,  this  would  not  lead  to  a significant  improvement. 

In  our  satellite  studies  we  permitted  satellite  access  only  in 
the  backbone  portion  of  the  network.  The  satellites  are  assumed  to 
operate  in  the  multiple  access  uplink,  broadcast  downlink  mode. 

With  our  particular  nominal  cost  assumptions  about  earth  stations,' 
they  would  not  be  economically  justified  in  the  local  access  area. 

In  designs  where  a backbone  node  has  terrestrial  connections  as  well 
as  satellite  connections  we  assumed  that  a regular  packet  switch  is 
required  in  addition  to  the  earth  station.  In  cases  where  the 
backbone  node  has  only  satellite  connections  to  the  other  backbone 
nodes,  we  assumed  the  presence  of  a device  - priced  at  the  largest 
concentrator  cost  - to  handle  the  local  access  lines  and  interface 
with  the  earth  station. 

A typical  network  topology  is  shown  in  Figure  1. 
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6.3  PARAMETERS  AND  COST  MODELS 


All  cost  parameters  are  either  expressed  on  a monthly  basis 
or  on  a capital  cost  basis.  Capital  costs  are  assumed  to  be 
reduced  to  an  equivalent  monthly  figure  by  multiplying  by  0.0438  - 
a factor  used  in  previous  NAC  studies  [NAC,  1975]  which  includes 
operation,  maintenance  and  amortization. 

6.3.1  Location  and  Traffic  Data 


The  data  base  used  for  the  design  studies  started  with  a 
purported  AUTODIN  II  projected  1980  requirement.  Requirements  to 
and  from  points  outside  the  U.S.  were  discarded.  Points  within 
the  U.S.  whose  requirements  and/or  location  were  unavailable 
(because  of  secrecy)  or  were  otherwise  unclear  were  also  deleted. 
These  adjustments  left  us  with  a data  base  with  272  points  whose 
distribution  is  still  very  representative  of  CONUS  AUTODIN  II  and 
hence  of  military  data  transmission  requirements  in  the  U.S.  in 
general. 

The  sums  of  the  input  and  output  requirements  had  a slight 
discrepancy  (less  than  1%)  and  the  input  requirements  for  all 
points  were  scaled  upward  slightly  to  achieve  a precise  balance. 
Since  the  objective  of  the  study  is  to  expose  major  trends,  and 
not  to  do  a detailed  design  for  implementation,  there  was  no  need 
to  refine  or  investigate  the  data  base  further.  The  exact  total 
data  throughput  requirement  came  out  to  be  1.833  MBS. 

6.3.2  Line  Parameters 

Real  world  leased  line  costs  are  complex  in  detail  and  have 
strange  dependencies  such  as  the  types  of  service  (e.g.,  DDS) 
currently  available  and  the. intricacies  of  access  arrangements. 
Many  of  these  matters  continue  td  be  volatile.  We  adopted  an 
approach  of  using  the  simplest  general  cost  structure  that  would 
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be  likely  to  reflect  accurately  the  gross  line  costs  at  any  time 
in  the  foreseeable  future. 

We  assumed  a simple  tariff  model  that  only  has  two  parameters, 
a fixed  line  charge,  F^,  independent  of  distance  and  a unit  mileage 
charge,  L^.  The  charge  can  be  assumed  to  include  all  termination 
charges  (including  modems,  local  loops,  etc.).  We  further  assumed 
that  the  line  charges  vary  with  the  square  root  of  the  bandwidth,  B 
(also  called  line  speed  in  digital  transmission) . This  yields  an 
approximately  40%  increase  in  line  cost  for  a doubling  of  bandwidth. 
This  latter  assumption  tracks  very  well  with  actual  costs  at  low 
and  intermediate  bandwidths  but  underestimates  line  costs  at  higher 
bandwidths,  as  represented  in  Figure  2.  This  situation  probably 
prevails  because  of  the  current  small  demand  for  very  high  (1.544 
MBS)  bandwidth  lines;,  but  physical  economies  of  scale  dictate  that 
the  square  root  relationship  will  be  at  least  approximately  true  in 
the  long  run.  Our  formula  for  line  cost  is  thus: 

Line  Cost  = (FL+LL  x mileage) /B. 

We  have  thus  removed  all  of  the  idiosyncrasies  of  existing 
tariffs.  To  make  the  problem  somewhat  more  realistic  we  retained 
the  complexity  of  having  only  a small  set  of  line  speed  choices 
reflective  of  the  current  options,  with  the  consequent  large  jumps 
between  options.  Although  large  jump  discrete  capacity  options 
greatly  increase  the  complexity  of  the  design  effort,  having  con- 
tinuous line  capacity  options  available  might  distort  the  applica- 
bility of  the  results.  The  speed  options  with  their  respective  end 
costs  and  mileage  charges  are  given  in  Table  1.  The  network  design- 
er and  design  programs  could  pick  multiple  lines  between  two  points 
in  the  backbone  portion  of  the  network  where  the  alternate  routing 
capability  reduces  the  sensitivity  of  the  network  to  sharp  design 
transitions  in  line  speed.  We  introduced  some  artificial  line 
speeds  in  the  local  access  portion  of  the  network  to  smooth  the 
otherwise  large  gaps  and  give  the  design  process  additional  stability 
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SPEED  (KBS) 

COST/MILE  ($/MO. ) 

COST/END  ( $/MO . ) 

1.2 

.30 

64 

2.4 

.42 

90 

00 

• 

.60 

125 

9.6 

.84 

175 

19.2 

1.20 

250 

50 

1.92 

400 

100* 

2.69 

560 

230 

4.22 

880 

460* 

5.97 

1244 

1544 

10.97 

2288 

♦Local  access  options  only. 


TABLE  1:  LINE  SPEED  OPTIONS 
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6.3.3  Backbone  Operation 


Since  the  design  methodology  adopted  (discussed  in  detail  in 
Section  6.4.1)  did  not  have  a fixed  backbone  delay  requirement,  there 
was  very  little  sensitivity  to  the  specific  operational  parameters 
selected.  For  example,  the  nodal  processing  time  per  packet  was 
assumed  to  be  a constant  1 ms.  , and  would  influence  only  the  actual 
delay,  but  have  absolutely  no  influence  on  the  relative  design  re- 
sults. A 7%  fraction  of  the  backbone  line  capacity  was  assumed  to 
be  devoted  to  activities,  such  as  routing  table  updates,  status 
monitoring,  etc. , which  are  network  overhead  operations  that  do  not 
depend  directly  on  traffic  volume. 

The  issue  of  the  appropriate  packet  protocol  overhead  factor, 
pQ,  is  somewhat  more  complex.  This  factor  accounts  for  such  items  . 
as  headers,  acknowledgments,  and  retransmission  (due  primarily  to 
errors)  which  are  traffic  related.  With  a fixed  bit  error  rate  - E, 
header  size  - H bits,  and  acknowledgment  size  - A bits,  pQ  is  a 
function  only  of  the  packet  size  distribution.  If  we  approximate 
by  assuming  all  packets  are  of  the  size  equal  to  the  average,  P, 
we  can  easily  compute  pQ  as  a function  of  P. 

We  make  this  more  explicit  with  the  following  first-order 
model:  we  assume  the  only  (primary)  effect  limiting  the  increase 

in  packet  size  is  the  resulting  overhead  from  retransmissions  due 
to  E;  we  assume  the  only  (primary)  effect  limiting  the  decrease 
in  packet  size  is  the  resulting  overhead  from  headers  and  hop-by- 
hop acknowledgments.  The  latter  overhead  is  simply 

H+A 
~ * 

The  probability  of  a correct  bit  is  1-E  so  the  probability  of  at 
least  one  error  in  a packet  of  size  P is 
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This  yields  a retransmission  overhead  of 


(1-E) 

The  total  overhead  is  thus 


(1-E) 


p - 1]  + 


Letting  E=10  , and  H=A=150  bits,  we  generate  the  following  values: 


P(bits)  [- 


(1-E) 


- 1] 


[H±A] 

i p J 


1000 

1500 

2000 

3000 

5000 


TABLE  2:  PACKET  PROTOCOL  OVERHEAD 

The  minimum  packet  overhead  is  achieved  in  the  neighborhood  of  P=1500 
bits.  Based  on  these  results  we  set  the  packet  protocol  overhead  to 
a nominal  value  of  35%.  Variations  in  the  value  of  E,  H,  A and  P 
would  impact  pQ  and  primarily  affect  the  backbone  design  and  cost. 
Design  sensitivity  to  variations  in  pQ  is  explored  later. 

The  average  packet  length  need  not  be  assigned  a specific 
nominal  value.  Different  assumptions  for  the  average  packet 
length  would  affect  the  absolute  delay  performance  of  the  networks 
obtained,  and  also  influence  their  relative  performance  if  an 
altered  average  packet  length  induces  a reassessment  of  the 
packet  protocol  overhead  factor  and/or  the  switch  costs.  The 
packet  protocol  overhead  factor,  pQ,  scales  the  backbone  traffic 
which  will  impact  the  required  throughput  in  the  backbone  network. 
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More  specifically,  a change  in  the  packet  size  would  scale  the 
line  delays  and  change  pQ  if  the  protocols  (headers,  ACK’s,  etc.) 
remained  constant.  Under  our  lack  of  specific  delay  criteria, 
scaling  of  the  line  delays  causes  an  equivalent  scaling  of  the 
end-to-end  delays  except  for  the  nodal  processing  delays  which 
remain  fixed.  The  nodal  processing  delays  were  only  a significant 
portion  of  the  end-to-end  delays  at  the  highest  traffic  levels 
studied.  A change  in  pQ  shifts  the  operating  point  on  the  through- 
put vs.  delay  curves  and  may  cause  the  accept/reject  decision  for 
a design  to  be  reversed. 

In  any  case,  the  designs  reported  in  the  next  section  were 
stable  for  all  packet  protocol  overhead  factors  in  the  neighborhood 
of  35%  (say,  at  least  through  40%) . In  summary,  because  of  our 
design  methodology,  the  assumption  on  packet  size  is  significant 
only  as  it  reflects  the  proper  range  of  overhead.  The  protocol 
overhead  factor  selectively  scales  the  backbone  traffic,  may 
influence  the  switch  throughput  cost  computation,  but  will  not 
affect  the  local  access  line  traffic  calculations. 

6.3.4  Switch  Parameters 

A simplified  model  was  adopted  for  the  cost  dependencies  of 
the  packet  switches.  We  assumed  the  cost  consisted  of  a fixed 
cost  component  and  a variable  cost  component  having  linear  dependence 
on  switch  throughput.  Switch  throughput  consists  of  two  components: 
the  local  traffic  that  has  both  its  origination  and  destination 
at  points  homed  to  the  same  backbone  switch,  and  which  therefore 
does  not  transit  the  backbone  links;  and  traffic  that  passes 
through  at  least  one  backbone  link  and  thus  contributes  to  the 
throughput  at  more  than  one  switch. 

The  switch  processing  capability  is  related  both  to  packet 
throughput  and  bit  throughput.  If  is  an  assumed  switch  capital 
cost  per  bit  of  throughput  capability,  Kj  is  an  assumed  switch 
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capital  cost  per  packet  of  throughput  capability,  and  F is  a 

s 

fixed  cost  per  switch,  we  have 

K2 

Switch  cost  = F + (K,  (1+p  ) + ^=-)  T 

S JL  u i 

where  T is  the  sum  of  the  local  traffic  and  backbone  traffic  in 
bits  per  second  without  overhead.  We  have  increased  the  local  bit 
traffic  component  by  the  overhead  factor  also  to  account  for  poll- 
ing (if  any),  header,  framing,  acknowledgments,  retransmissions, 
and  other  facets  of  local  access  protocols  not  reflected  in  the 
basic  site  I/O  requirements  data.  P and  pQ  are  not  independent  and 
are  themselves  functions  of  other  parameters  as  shown  in  the  last 
subsection.  We  can  simplify  this  cost  equation  by  lumping  all 
the  parameters  in  the  coefficient  of  T into  a single  linear  switch 
throughput  cost  factor,  Lg,  and  add  a scale  factor,  S,  nominally 
set  equal  to  one,  that  permits  networks  to  be  appraised  at  differ- 
ent line  cost-to-switch  cost  ratios. 

Our  simplified  cost  calculation  for  switches  is  thus 

Switch  Cost  = S (Fg+LgT) . 

The  nominal  values  used  in  this  study  are  Fs=$100,000  and  Lg=$. 1/BPS 
so  that  a switch  with  one  MBS  throughput  has  a cost  of  $200,000. 

Values  of  S less  than  one  were  applied  to  these  costs. 

Note  that  with  this  simple  switch  cost  relationship  we  can 
lump  all  the  switches  together;  multiply  the  number  of  switches 
by  Fg  and  add  the  total  throughput  (local  plus  backbone  at  each 
switch)  multiplied  by  the  Lg  factor  to  get  the  unsealed  switch  cost. 
Thus  given  only  two  parameters,  the  number  of  backbone  switches,  Ng, 
and  the  sum  of  the  T's  for  the  switches,  we  can  price  out  the  switch 
costs  of  a given  network  under  a variety  of  switch  cost  assumptions. 
Additional  complexity  - not  considered  in  this  report  - would  be 
introduced  by  pricing  switches  in  different  size  ranges  with  different 
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parameters,  ultimately  going  to  discrete  pricing.  This  would 
make  repricing  a network  under  different  technology  assumptions  a 
chore. 

An  important  point  to  note  is  that ; when  the  number  of  backbone 
sites  is  increased,  in  general,  we  have  a correlated  increase  in 
backbone  throughput.  A smaller  portion  of  the  traffic  is  likely 
to  be  local  to  a switch  and  the  average  number  of  backbone  hops 
traversed  between  sources  and  destinations  increases  since  the 
average  node  degree  does  not  increase  proportionately.  We  note 
again  that  T is  calculated  independently  of  packet  overhead  so 
there  is  no  additional  cost  penalty  due  to  overhead  when  traffic 
switches  from  local  to  backbone  traversing  as  additional  backbone 
nodes  are  introduced. 

6.3.5  Concentrator  Parameters 


We  assumed  the  existence  the  three  generic  concentrator 
models  designed  to  handle  low  (voice  grade  or  below) , wideband, 
and  extra  wideband  requirements  as  presented  in  Table  3 below. 

Throughput  Capacity  (KBS)  Nominal  Cost  ($) 

9.6  10,000 

50  50,000 

230  100,000 

TABLE  3 : CONCENTRATORS 

The  hardware  cost  scale  factor,  S,  was  applied  to  the  concen- 
trator costs  as  well  as  the  switch  costs,  so  the  table  above  rep- 
resents the  highest  costs  used  in  the  study.  No  restrictions  were 
placed  on  the  number  of  ports  available  at  a particular  concentrator. 
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6.3.6  Local  Access  Utilization 

Sites  engaged  exclusively  in  batch  or  large  file  transfer  oper- 
ations can  get  very  high  utilization  of  their  local  access  lines 
without  performance  degradation.  For  interactive  users,  the  utili- 
zation has  a significant  impact  on  delay  performance.  Since  we  did 
not  have  a characterization  of  the  traffic  type  composition  for  the 
individual  sites,  and  to  retain  the  formulation  simplicity,  we 
adopted  a uniform  rule  of  limiting  the  steady  state  traffic  on  any 
access  line  to  65%  of  its  capacity.  This  assures  good  performance 
stability  even  for  highly  random  interactive  traffic.  The  choice  of 
values  other  than  65%  would  be  roughly  equivalent  to  scaling  local 
access  costs  relative  to  other  costs  in  its  impact  on  the  design. 

As  pointed  out  before,  individual  users  could  selectively  upgrade  or 
degrade  their  access  lines  with  no  significant  effect  on  the  rest  of 
the  network. 

6.3.7  Satellite 

For  designs  using  a satellite  we  assumed  multiple  access 
uplinks  and  broadcast  mode  downlinks.  Each  earth  station  was 
assumed  to  nominally  cost  $1,000,000  (equivalent  to  $43, 800/month 
for  operation,  maintenance  and  amortization) . The  satellite 
bandwidth  was  assumed  to  be  allocated  in  units  of  1.544  MBS  - we 
are  only  concerned  here  with  the  high  throughput  range.  We  again 
assumed  a square  root  increase  of  cost  versus  capacity,  with  a 
nominal  cost  of  $30 , 000/month  for  a single  1.544  MBS  channel. 

The  cost  is  assumed  to  absorb  any  proportional  additional  bandwidth 
needed  to  support  a satellite  protocol.  Thus  for  a throughput, 

T,  through  the  satellite  the  nominal  cost  is 

Nominal  Satellite  Channel  Cost  = 3x10^ 
where  N is  the  smallest  integer  satisfying  N _>  T/1.  544  MBS. 
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For  a single  channel  this  cost  is  equivalent  to  a 2290  mile  terres- 
trial channel  of  the  same  bandwidth.  Scale  factors  less  than  one 
were  subsequently  applied  to  the  satellite  channel  costs. 
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6.4  DESIGNS 


6.4.1  Methodology 


6. 4. 1.1  Backbone  Assignments  and  Traffic  Matrix 

Since  we  did  not  have  a source-destination  traffic  matrix 
for  the  272  sites  involved  in  our  study,  we  adopted  the  following 
technique  to  obtain  a backbone  traffic  matrix. 

1.  A set  of  backbone  node  locations  is  determined 
by  a combination  of  automatic  and  interactive 
techniques. 

2.  Local  sites  are  assigned  to  the  backbone 
nodes  on  some  automatic  and/or  interactive 
basis.  Usually  the  rule  used  was  to  assign  a 
site  to  the  closest  backbone  node. 

3.  The  total  transmitted  traffic  and  total 
received  traffic  from  a backbone  node  to  its 
assigned  local  sites  was  calculated  from  the 
local  site  I/O  requirements  data. 


4.  The  total  transmitted  traffic  for  each  back- 
bone node  was  assigned  to  be  received  by  the 
other  backbone  nodes  and  itself  in  the  propor- 
tion to  which  each  node's  total  received 
traffic  was  a fraction  of  the  entire  network's 
received  traffic.  Thus,  for  example,  a backbone 
node  whose  assigned  local  sites  receive  25% 
of  the  total  received  traffic  from  all  network 
sites,  would  be  assumed  to  receive  its  share 
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of  25%  of  the  traffic  transmitted  by  every 
other  node.  Also  25%  of  its  own  transmitted 
traffic  is  assumed  not  to  require  transiting 
the  backbone  links;  this  is  traffic  local  to 
the  switch.  In  other  words,  sources  distribute 
their  traffic  to  destinations  in  direct  propor- 
tion to  the  magnitude  of  each  destination's 
relative  receive  requirements.  (See  Figure  3) . 

The  above  method  is  a reasonable  assumption  in  the  face  of 
lack  of  specific  data.  It  is  also  well  established  that  the  back- 
bone design  is  very  sensitive  to  overall  traffic  levels,  but  not, 
in  general,  sensitive  to  the  specific  traffic  distribution. 

Note  that  with  the  same  site  traffic  levels,  the  backbone 
throughput  will  depend  somewhat  on  the  local  site-to-backbone  node 
connections  and,  in  particular,  will  be  affected  by  the  number  of 
backbone  nodes  selected. 


6. 4. 1.2  Design  Acceptance 


Since  one  of  our  objectives  is  to  study  design  variations  as 
the  traffic  level  is  varied  over  a wide  range,  it  was  not  possible 
to  compare  networks  on  the  basis  of  equivalent  backbone  delays. 

The  backbone  link  capacities  are  determined  largely  by  throughput 
requirements,  and  this  could  vary  considerably  even  at  the  same 
overall  traffic  level  as  the  number  of  backbone  nodes  in  the 
design  changes. 

Consequently,  for  each  traffic  scale  factor,  hardware  scale 
factor,  and  transmission  media  selection,  we  sought  to  find  the  best 
(cheapest)  obtainable  acceptable  network  by  rejecting  networks  that 
could  not  satisfy  the  throughput  requirements  at  a stable  point  on 
their  throughput  versus  delay  curves.  Although  "stability"  in  the 
sense  used  here  is  somewhat  subjective,  it  offered  no  significant 
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FIGURE  3: 


SCHEMATIC  OF  BACKBONE  TRAFFIC  DISTRIBUTION  ASSUMPTION 
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obstacles  in  practice.  Since  the  possible  line  capacities  are 
chosen  from  a small  discrete  set,  well  designed  networks  will 
often  shift  radically  from  clearly  acceptable  to  clearly  unacceptable 
if,  among  the  discrete  choices  available,  smaller  (and  less  costly) 
capacities  are  substituted. 

The  designs  obtained  as  "best"  are  usually  surrounded  by 
similar  costing  designs,  often  with  quite  different  topologies, 
and  thus,  in  a sense,  are  representatives  of  a whole  class  of 
good  designs.  The  designs  reported  are  not  strictly  guaranteed 
to  be  optimal,  but  represent  the  best  that  can  be  obtained  under 
current  technology  with  reasonable  effort. 

The  selection  of  a single  "best"  design  for  one  design 
variable  point  may  have  involved  the  generation  of  literally 
hundreds  of  alternate  designs. 

6. 4. 1.3  Study  Variables 

The  basic  study  variables  were  explored  as  described  below: 

1.  Traffic  level  - Sets  of  designs  were  obtained 

for  three  traffic  levels:  1.83  MBS  (nominal), 

18.3  MBS  (10  times  nominal),  1.83  MBS  (100 
times  nominal) . 

2.  Cost  scale  factor,  S - Four  different  ratios  of 

hardware  cost  to  line  cost  were  explored  for 
each  traffic  level:  S=1  (nominal),  S=l/2, 

S=l/5,  S=l/10. 

3.  Number  of  backbone  switches  - The  most  important 
decision  alternative  and  design  variable  under 
the  architecture  adopted  is  the  number  of  back- 
bone switch  sites.  For  each  traffic  level  and 
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cost  scale  factor,  best  designs  were  obtained 
for  the  range  of  two  to  eight  backbone  sites. 
Where  trends  were  not  already  obvious,  the 
range  was  extended  to  designs  with  12  and  25 
backbone  switch  locations. 

Dual  homing  - The  added  cost  of  dual  homing 
concentrators  to  an  alternate  backbone  switch 
- without  redesign  - were  determined  for  the 
lowest  traffic  level  (full  range  of  cost  ratios 
and  number  of  backbone  nodes) . Concentrators 
were  not  significant  in  the  designs  at  other 
traffic  levels  and  thus  neither  would  be  the 
dual  homing  costs. 

Satellite  usage  - The  satellite  studies  were 
confined  to  the  18  MBS  traffic  level  case  and 
the  trends  exposed  should  be  reflective  of 
what  would  occur  at  the  other  traffic  levels. 
Designs  were  generated  for  the  range  of  number 
of  backbone  nodes  for  the  case  of  only  satellite 
links  in  the  backbone  network;  each  backbone 
site  is  assumed  equipped  with  an  earth  station 
for  satellite  access.  Progressively  lower 
satellite  channel  costs  and  earth  station 
costs  were  applied  to  the  designs  obtained. 

Then  for  a representative  fixed  number  of 
backbone  nodes,  designs  were  obtained  for 
various  numbers  of  earth  stations  under 
two  different  assumptions: 

a.  Satellite  connectivity;  backbone  sites 
without  earth  stations  home  to  nearest 
earth  station. 
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b.  Mixed  satellite  and  terrestrial  con- 
nectivity; guaranteed  minimum  50  KBS 
two-connected  terrestrial  "thin-line" 
path  between  any  pair  of  backbone  nodes. 

6.  Packet  protocol  overhead  - Changes  in  pro- 
tocols, packet  size,  or  both  will  effect 
the  packet  protocol  overhead.  The  primary 
design  effect  will  be  altered  capacities 
and/or  topology  in  the  backbone  links. 

Switch  costs  will  also  be  effected  and  may 
shift  the  optimum  number  of  switches.  There 
will  be  no  affect  on  the  local  access  design. 
Switch  locations  will  be  affected  for  large 
swings  in  overhead  since  this  selectively 
scales  the  backbone  line  costs. 

We  studied  the  effect  of  a wide  range  of 
overheads  on  the  backbone  network  needed 
to  support  the  traffic  from  a fixed  set  of 
5 switch  locations  for  each  of  the  three 
traffic  levels. 

6.4.2  Results 


The  results  below  are  given  both  tabularly  and  graphically. 

They  were  obtained  using  the  parameters  stated  in  Section  6.3  with 
the  methodology  given  in  Section  6.4.1.  The  latest  version  of  NAC's 
interactive  design  software  was  used  throughout  as  the  main  design 
tool,  supplemented  by  human  designer  judgment. 

Most  of  the  designs  obtained  - particularly  the  backbone  node 
locations  - would  vary  little  and  insignificantly  under  a variety 
of  alternate  parameter  assumptions.  Enough  information  is 
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presented  in  tabular  results  below  to  "unpack"  the  design  properties 
and  vary  the  assumptions.  Thus,  for  example,  knowing  the  number 
of  backbone  sites  and  total  backbone  switch  cost,  one  could 
calculate  the  total  throughput,  T,  and  then  cost  out  the  switches 
under  different  assumptions.  Different  switch  costs  would  have  a 
big  effect  on  the  optimal  number  of  switches  but  would  not  have 
much  impact  on  optimal  location  of  a given  number  of  switches. 

The  best  locations  are  primarily  determined  by  a tradeoff  of 
backbone  line  cost  and  local  access  cost  - the  former  carrying 
traffic  more  directly,  the  latter  carrying  traffic  less  directly 
but  in  higher  bandwidth  lines. 

6. 4. 2.1  Terrestrial  Designs 

Figures  4-15  and  Tables  4-15  present  the  basic  terrestrial 
design  costs  for  the  different  traffic  levels  and  hardware  cost 
scale  factors  as  the  number  of  backbone  nodes  is  varied. 

Figures  and  Tables  16-19  show  the  effect  of  added  dual 
homing  costs  to  the  networks  of  nominal  traffic  level  and  hardware 
cost  (Figures  and  Tables  4-7) . Note  that  as  the  number  of  backbone 
nodes  increase  the  dual  homing  costs  quickly  die  off.  This 
occurs  because  the  cost  justification  for  local  access  concentrators 
disappears  as  more  backbone  nodes  are  distributed  and  also  because 
the  cost  of  the  dual  homed  line  gets  smaller  as  closer  alternate 
backbone  sites  become  available.  At  higher  traffic  levels  this 
effect  was  even  more  pronounced  with  dual  homing  costs  virtually 
disappearing  when  3 or  more  backbone  switches  became  available 
for  the  case  of  10  times  nominal  traffic. 

6. 4. 2. 2 Satellite  Designs 


Table  20  shows  the  results  of  replacing  the  backbone  portion 
of  the  networks  that  carried  10  times  the  nominal  traffic  with 
satellite  links.  The  number  of  1.544  MBS  satellite  channels 
needed  to  support  the  packet  throughput  (with  the  allowance  for 
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satellite  protocol  overhead  reflected  through  the  cost  factor)  is 
shown  explicitly.  Each  $1,000,000  ground  station  is  assumed  to 
require  a $100,000  local  access- to-ground  station  interface 
device.  Local  access  costs  otherwise  remain  exactly  as  they  were 
for  the  nominal  hardware  cost  terrestrial  case.  Small  cost 
reductions  ( 1%)  might  be  achievable  in  the  local  access  area  by 
relocating  the  backbone  nodes.  These  were  located  for  best 
overall  cost  in  the  terrestrial  design  where  some  consideration  - 
though  small  - had  to  be  given  to  the  distance  dependent  terrestrial 
backbone  line  cost.  To  obtain  this  small  perturbation  would  have 
involved  an  expensive  reoptimization  of  all  the  local  access 
designs  and  hence  was  not  done.  Taking  the  backbone  locations 
(and  hence  the  local  access  costs)  to  be  fixed,  the  backbone 
portion  can  be  recosted  for  any  alternate  satellite  assumptions 
using  the  parameters  N and  the  number  of  backbone  sites.  The 
costs  in  Table  20  have  been  computed  using  the  nominal  parameters 
of  Section  6.3.7  and  are  plotted  on  Figure  20.  The  results  for 
the  same  case  with  terrestrial  backbone  links  is  plotted  for 
convenient  comparison.  Tables  and  Figures  21-23  show  the  effect 
of  progressively  reducing  the  satellite  channel  cost  on  the 
designs  with  only  satellite  backbone  connections.  Fixing  the 
satellite  channel  cost  at  1/2  nominal.  Tables  and  Figures  24-26 
show  the  effect  of  reducing  the  hardware  cost,  now  including  the 
ground  station  and  interface  cost,  on  the  backbone  and  total  net- 
work cost. 

For  our  next  study,  we  froze  the  backbone  node  positions  of 
the  5 switch  network  for  the  case  of  traffic  10  times  nominal  and 
nominal  hardware  cost.  We  then  proceeded  to  optimally  select 
different  size  subsets  of  the  5 nodes  to  have  ground  stations. 

The  remaining  backbone  nodes,  if  any,  were  homed  to  the  nearest 
ground  station.  Thus  network  interface  devices  (costed  at  largest 
concentrator  cost)  were  required  at  each  backbone  node.  The 
results  are  tabulated  in  Table  27a.  Note  that  no  switch  costs 
are  involved  and  that  the  total  costs  refer  to  backbone  costs 
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only.  The  5 ground  station  case  corresponds  to  the  5 backbone 
node  design  in  Table  20  (only  satellite  links  in  backbone  network) 
and  the  0 ground  station  case  in  Tables  27a  and  27b  corresponds 
to  the  5 backbone  node  pure  terrestrial  link  case  of  Table  8. 

The  costs  for  these  two  extreme  cases  also  appear  in  Figure  20  as 
the  backbone  network  costs  (lower  two  curves)  at  the  5 backbone 
node  level. 

For  comparison  we  also  designed  networks  for  the  same  situation 
but  with  an  additional  requirement  of  maintaining  at  least  50  KBS 
two-connected  terrestrial  path  connectivity  (for  backup,  high 
priority,  or  low  delay  traffic)  between  all  five  backbone  nodes. 

This,  of  course,  now  required  switches  at  these  backbone  sites. 

The  results  are  tabulated  in  Table  27b.  The  backbone  cost  results 
of  Tables  27a  and  27b  are  plotted  for  comparison  in  Figure  27. 

We  next  sought  to  investigate  the  sensitivity  of  our  designs 
and  results  to  some  critical  network  model  assumptions  and  parameters. 
In  Tables  and  Figures  28  through  31  we  show  the  effect  of  providing 
an  additional  line  option,  6.176  MBS  (4  x 1.544  MBS),  in  the  100 
times  nominal  traffic  case  across  all  hardware  cost  scale  factors. 
Tables  and  Figures  32-34  show  the  effect  of  varying  the  packet 
protocol  overhead,  pQ,  on  the  backbone  line  cost  for  the  networks 
with  5 backbone  nodes  at  the  various  traffic  levels. 
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6.5.  CONCLUSIONS 


Tables  35  and  36  show  the  best  network  cost  obtained  and  the 
number  (or  range,  if  precise  number  was  not  decidable  from  the 
study)  of  backbone  nodes  in  the  best  terrestrial  configuration. 
From  these  summary  tables  and  the  supporting  data  we  can  draw  the 
following  conclusions  for  the  whole  range  of  hardware  cost  scale 
factors. 

1.  Line  costs  are  the  dominant  cost  factor  with 
local  access  costs  dominating  over  backbone 
line  costs.  The  only  slight  exceptions  are: 

a.  With  traffic  scaled  100  times  and 
nominal  hardware  cost,  the  switch 
costs  amounted  to  almost  30%. 

b.  With  traffic  scaled  100  times  and 
with  more  than  5 backbone  nodes, 
the  local  access  and  backbone  line 
costs  became  comparable. 

2.  From  Table  35  we  see  that  the  network  costs  in- 
crease by  a factor  of  about  3.4  when  the  traf- 
fic is  scaled  to  10  times  nominal.  This  is 
primarily  due  to  the  square  root  (of  10)  in- 
crease in  line  cost  with  bandwidth. 

3.  Again  from  Table  35  we  see  that  network  costs 
increase  by  a factor  of  6 as  traffic  is  scaled 
from  10  times  nominal  to  100  times  nominal. 

This  is  due  to  several  factors  but  primarily: 
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Hardware  Cost 


Traffic 

Nominal 

1/2 

1/5 

1/10 

Nominal 

255 

235 

220 

213 

10  times 

883 

793 

739 

712 

100  times 

5443 

4686 

4231 

4080 

100  times 

4692 

4006 

3421 

3227 

(w  6 MBS  lines) 


TABLE  3 5 : TERRESTRIAL  NETWORK  COST 


Hardware  Cost 


Traffic 

Nominal 

1/2 

1/5 

1/10 

Nominal 

5 

5 

5 

5 

10  times 

4-5 

5 

5-6 

5-12 

100  times 

2-5 

5 

5 

5 

100  times 

5 

4-8 

8- 

8- 

(w  6 MBS  lines) 


TABLE  36:  OPTIMAL  NUMBER  OF  BACKBONE  NODES 


Note:  Costs  for  all  tables  expressed  in  thousands  of  dollars. 
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a.  Switch  costs  are  becoming  a more  signif- 
icant factor  and  we  have  assumed  a portion 
of  the  switch  cost  to  vary  linearly  with 
throughput. 

b.  To  obtain  the  needed  large  line  capacities, 
the  links  often  required  multiples  of  1.544 
MBS  channels.  Since  this  capacity  is  the 
largest  assumed  available  terrestrial  option, 
increases  in  bandwidth  have  a linear  depen- 
dence on  bandwidth  when  such  multiples  are  re- 
quired. In  effect,  we  have  situations  that 
exceed  the  range  of  bandwidth  economies  of 
scale.  Under  current  tariffs  this  point 
begins  at  lower  bandwidths  than  we  modeled 
under  our  line  cost  formulation.  Nonethe- 
less, extending  the  range  of  square  root 
economies  of  scale  from  1.544  MBS  to  6.176 
MBS  reduced  the  cost  factor  increase  of 

the  second  10  times  traffic  increase  to  the 
4 to  5 range. 

4.  In  all  cases,  the  first  few  switches  introduced 

sharply  reduced  costs  by  making  a favorable  trade- 
off between  reduced  local  access  costs  and  backbone 
line  costs.  This  is  usually  followed  by  a flat 
portion  of  the  total  network  cost  curve  as  this 
favorable  tradeoff  is  ameliorated  by  an  increasing 
switch  cost  component.  As  more  switches  are  intro- 
duced a point  is  reached  where  decreasing  local 
access  costs  and  increasing  backbone  line  costs 
cancel  out  and  the  increasing  switch  cost  compon- 
•ntukcs  these  networks  uneconomical.  Decreasing 
* h.  hardware  cost  factor  extended  the  flat  portion 
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to  the  right  (more  switches) . The  use  of 
concentrators  in  the  local  access  area  ex- 
tended the  flat  portion  to  the  left  (fewer 
switches)  particularly  at  the  nominal  traffic 
levels.  Each  of  these  last  two  factors  also 
reduced  the  slope  of  the  "rise"  at  the  respec- 
tive ends  of  the  flat  region. 

5.  In  all  cases,  reducing  the  hardware  cost  scale 
factor  suppressed  the  right-hand  rise  from  the 
flat  portion  - reducing  the  significance  of 
switch  cost.  Because  of  the  geographical  idio- 
syncrasies of  the  distribution  of  military 
installations,  5 backbone  nodes  turned  out  to 
be  a particularly  good  number  of  backbone  nodes. 

Note  in  particular  that  at  the  10  times  nominal 
traffic  level,  there  is  only  a small  dependence 
on  the  correct  selection  of  switch  number.  At 
the  1/10  hardware  cost  scale  there  was  less 
than  a 1%  variation  in  total  network  cost  for 
the  range  of  5-12  backbone  nodes  - an  almost 
total  lack  of  decision  criticality  for  this 
traffic  range  and  hardware  cost!  However, 
at  the  180  MBS  throughput  level  the  optimal 
number  of  backbone  nodes  shifted  back  to  the 
lower  range,  with  5 again  emerging  as  the 
best  choice  independent  of  hardware  cost. 

The  best  decision  range  is  now  in  the  region 
of  a few  - 5 or  less  - backbone  nodes.  Under 
the  extended  range  of  bandwidth  economies, 
the  optimum  number  of  nodes  shifted  to  the  8 
or  over  region  for  most  hardware  cost  scale 
factors. 

An  important  conclusion  is  that  providing  a number  of  backbone 
beyond  a small  region  around  five,  is  only  economical  for  the 
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middle  range  of  traffic  (10  times  nominal)  and  low  hardware  costs 
(1/10  nominal)  unless  strong  bandwidth  economies  of  scale  continue 
to  pertain  as  high  as  6 MBS. 

6.  The  costs  of  dual  homing  concentration  points 
shifts  the  optimal  region  from  around  5 back- 
bone nodes  to  around  8 backbone  nodes  for  the 
nominal  traffic  case.  There  is  no  other  sig- 
nificant effect  for  the  other  traffic  cases 
under  our  assumption  of  limited  concentrator 
traffic  handling  ability. 

We  can  draw  the  following  conclusions  from  Tables  and  Figures 
20-23  which  compared  pure  satellite  backbone  networks  with  terrestrial 
backbone  networks  both  at  nominal  hardware  costs: 

7.  The  optimum  number  of  backbone  nodes  has  shifted 
to  a lower  number  (3  in  this  case)  - the  result 
of  high  earth  station  costs. 

8.  There  is  a cheaper  design  with  all  satellite  back- 
bone links  than  any  design  with  all  terrestrial 
backbone  links  - the  result  of  high  bandwidth 
concentration. 

9.  Each  pure  backbone  network,  terrestrial  and 
satellite,  has  a distinct  range  of  number  of 
switches  for  which  it  is  more  economical. 

10.  With  nominal  earth  station  costs,  even  radical 
cost  reductions  (1/10)  in  satellite  bandwidth 
did  not  dramatically  affect  total  network  cost 
or  shift  the  optimum  number  of  earth  stations. 

The  criticality  of  the  number  of  earth  stations 
did  reduce  and,  of  course,  the  relative  advantage 
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of  the  satellite  backbone  over  the  terrestrial 
did  increase.  With  the  reduced  satellite 
channel  cost,  the  satellite  backbone  was  cheaper 
for  networks  with  fewer  than  8 backbone  nodes. 


Figures  and  Tables  24-26  show  the  importance  of  earth  station 
cost  in  network  strategies. 


11. 


With  dropping  earth  station  cost  the  satellite 
designs  remained  cheaper  than  terrestrial  up  to 
about  25  backbone  nodes  for  1/2  cost,  to  beyond 
25  nodes  for  1/5  cost,  and  for  the  1/10  cost 
case  (earth  station  cost  is  reduced  to  $100,000) 
the  costs  were  still  actually  dropping  at  25 
nodes,  indicating  the  merit  of  extending  satellite 
access  to  the  local  access  area. 


From  Figure  and  Table  27  we  see  that  by  redesigning  the  5 
backbone  configuration  allowing  both  satellite  and  terrestrial 
links  we  conclude: 


? * 

Sr 


12.  Expectedly,  cheaper  networks  result  when  the 

designer  is  not  restricted  to  either  satellite 
or  terrestrial  2 inks.  In  this  case  the 
cheapest  backbone  network  occurred  with  2 
ground  stations  (see  lower,  curve  of  Figure  27) 
and  3 ground  stations  also  resulted  in  a 
slightly  cheaper  network  than  either  for  the 
"pure"  networks  (the  end  points  of  the  lower 
curve  of  Figure  27).  Note  that  with  4 and  5 
ground  stations  the  best  designs  were  more 
expensive  than  the  pure  terrestrial  case. 


It 
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13.  The  upper  curve  of  Figure  27,  when  compared  to  the 
lower  curve,  demonstrates  the  high  added  cost  of 
maintaining  terrestrial  connectivity  is  a mixed 
terrestrial/satellite  network. 

In  Tables  and  Figures  28  to  31  we  extended  the  range  where 
the  "square  root"  economies  of  scale  with  bandwidth  apply  to  the 
6.176  MBPS  range  and  re-examined  our  design  results. 

14.  Backbone  line  costs  remain  relatively  flat 
for  a greater  range  of  backbone  nodes  making 
designs  with  5-12  backbone  nodes  generally 
more  attractive. 

From  Tables  and  Figures  32-34  we  can  observe: 

15.  The  sensitivity  of  terrestrial  backbone  line 
cost  to  variations  in  pQ,  the  packet  protocol 
overhead,  is  approximately  proportional  to 
the  ratio  of  (1  + p )/1.35,  where  a pQ  of 
35%  is  used  as  a base  index.  Thus  the  cost 
of  any  terrestrial  network  design  can  be 
recosted  with  this  approximation  for  any 
assumption  on  pQ.  Recall  that  pQ  may,  in 
general,  also  affect  the  throughput  cost 
coefficient  of  the  switch  cost. 

This  study  was  supported  by  a highly  developed  interactive 
software  tool  developed  at  NAC  under  ARPA  sponsorship  which  allowed 
us  to  obtain  well  designed  networks  at  reasonable  computation  cost 
as  we  varied  a number  of  significant  variables  and  options. 

The  most  significant  conclusion  is  that  a relatively  small 
number  of  backbone  nodes  (say,  5)  appears  to  be  the  best  strategy 
for  a wide  range  of  cost  and  traffic  assumptions  for  packet-switched 
networks. 
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